<html>
<head><meta charset="utf-8"><title>Improved span handling #47389 · t-compiler · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/index.html">t-compiler</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html">Improved span handling #47389</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="196115074"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/196115074" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cjgillot <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#196115074">(May 03 2020 at 18:04)</a>:</h4>
<p>I am trying to make progress on Span handling for incremental compilation.<br>
The point of this thread is to gauge interest, and to find the least painful way.</p>
<p>In the current state, spans are copied and replicated in AST, HIR and MIR. This leads to invalidation when the spans change.<br>
<span class="user-mention" data-user-id="124287">@mw</span> outlined two paths forward:</p>
<blockquote>
<p>These goals can be achieved by:</p>
<ul>
<li>Splitting out span information during HIR lowering and storing it separately. I imagine having one table per HirId::owner that then corresponds to one DepNode, making spans be tracked at the same granularity as HIR items.</li>
<li>Replacing Span fields with abstract spans in HIR, MIR, etc. This will mean quite a bit of refactoring everywhere these spans are used (as opposed to just being copied around)</li>
</ul>
<p>Alternatively, this could also be achieved by:</p>
<ul>
<li>Making the CodeMap inaccessible from queries and generating a map from Span value to DepNode during HIR lowering, thus effectively making the existing Span type abstract.</li>
<li>Providing a query that allows to decode a Span to its contents.</li>
<li>Making sure that none of the error reporting APIs take Spans directly.</li>
</ul>
</blockquote>
<p>Is there still interest for this?<br>
What would be the preferred solution?</p>



<a name="196296772"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/196296772" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#196296772">(May 05 2020 at 11:48)</a>:</h4>
<p>I think this is probably an important thing to work on -- I'm not sure the best design though</p>



<a name="196296781"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/196296781" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#196296781">(May 05 2020 at 11:48)</a>:</h4>
<p>it'd be good to bring <span class="user-mention" data-user-id="133169">@matklad</span> into the conversation a bit, I think</p>



<a name="196297484"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/196297484" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#196297484">(May 05 2020 at 11:56)</a>:</h4>
<p>Yup, we are solving the same problem in rust-analyzer. I belive <span class="user-mention silent" data-user-id="282393">Coenen Benjamin</span> is familiar with our general infra. </p>
<p>I wonder if I should spend some time writing up a markdown document documenting our various solutions to the problem of bidirectional mapping between source code and various lowered representations</p>



<a name="196299455"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/196299455" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#196299455">(May 05 2020 at 12:18)</a>:</h4>
<p>This seems like a case where it'd be nice to be trying to migrate the two codebases closer</p>



<a name="196304762"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/196304762" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#196304762">(May 05 2020 at 13:08)</a>:</h4>
<p>And also the case where even the basic vocabulary is different. Like, we don't have anything resembling span in rust-analyzer :)</p>



<a name="196339072"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/196339072" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cjgillot <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#196339072">(May 05 2020 at 17:24)</a>:</h4>
<p>I have two exploratory branches. </p>
<p>The first one makes the error handling system take an abstract span <code>SpanId</code>. There are some diagnostics that just pass around a span without manipulating it. The idea is to make an <code>enum SpanId { Span(Span), DefSpan(DefId) };</code>. The calls to <code>tcx.def_span(did)</code> would just return <code>SpanId::DefSpan(did)</code>. Diagnostic tracking would register emitted diagnostics using this abstract representation, and translate it to concrete spans. </p>
<p>The second one attempts at removing the Spans from the HIR. The spans are collected during lowering into a <code>HirId -&gt; Span</code> mapping. HIR consumers would need to query this map in order to formulate diagnostics.</p>
<p>Both approaches are invasive, and touch a <em>lot</em> of code. All I have is very WIP. This maybe requires a review of uses of Spans in the compiler. Especially, when those spans end up not being used for diagnostic emission.</p>



<a name="196988946"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/196988946" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cjgillot <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#196988946">(May 09 2020 at 12:14)</a>:</h4>
<p>I posted <a href="https://github.com/rust-lang/rust/issues/72015" title="https://github.com/rust-lang/rust/issues/72015">#72015</a> which aims at removing the Spans from the HIR. The point is to reduce HIR invalidation by span modification as much as possible. For now, only nodes that have <code>HirId</code>s have been migrated, and its already a mess.</p>



<a name="196989712"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/196989712" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#196989712">(May 09 2020 at 12:35)</a>:</h4>
<p>I'm going through the commits now, what do you mean by "it's already a mess"?</p>



<a name="196992352"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/196992352" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cjgillot <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#196992352">(May 09 2020 at 13:45)</a>:</h4>
<p>Well, the diff is horribly large (+3000/-2700), and touches pretty much all HIR consumers. At some point, I did it mechanically, and stopped caring for redundant accesses to the <code>HirId -&gt; Span</code> map.</p>



<a name="197123496"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/197123496" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#197123496">(May 11 2020 at 10:04)</a>:</h4>
<p>so you're worried about perf losses due to the redundant accesses?</p>



<a name="197123640"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/197123640" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#197123640">(May 11 2020 at 10:06)</a>:</h4>
<p>I'd expect the diff for the full McCoy to be large, but as you've shown, this can be done incrementally, so maybe we should run perf on your change and see if it's ok, and if so, merge a small version and then the rest of the changes can be merged step by step. I think we'll also need to figure out how to handle spans in MIR at some point.</p>



<a name="197156566"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/197156566" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cjgillot <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#197156566">(May 11 2020 at 14:53)</a>:</h4>
<p>The point is to get better perf in incremental, so we shouldn't be careless in the accesses. Also, there are still a lot of spans in HIR, for nodes without <code>HirId</code>s. As I am not fully aware of the semantics of HirIds, I did not try to introduce more. Is there a blocker to adding HirIds everywhere?</p>



<a name="197175904"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/197175904" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#197175904">(May 11 2020 at 17:08)</a>:</h4>
<p>I think not, there was the "great hirification": <a href="https://github.com/rust-lang/rust/issues?q=hiridification+is%3Aclosed+">https://github.com/rust-lang/rust/issues?q=hiridification+is%3Aclosed+</a> so... Let's ask the @ljedrz (they aren't on zulip it seems <span aria-label="confused" class="emoji emoji-1f615" role="img" title="confused">:confused:</span>)</p>



<a name="197175977"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/197175977" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oli <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#197175977">(May 11 2020 at 17:09)</a>:</h4>
<p>maybe open an issue about it and/or ping them on the PR?</p>



<a name="197178911"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/197178911" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dylan MacKenzie (ecstatic-morse) <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#197178911">(May 11 2020 at 17:33)</a>:</h4>
<p>I think one blocker was <a href="https://github.com/rust-lang/rust/issues/71104">#71104</a>, but it's actively being worked on. Thanks <span class="user-mention" data-user-id="281572">@marmeladema</span>!!!</p>



<a name="197179503"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/197179503" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#197179503">(May 11 2020 at 17:37)</a>:</h4>
<p>Yes i am trying to make progress! But as I am a new contributor, i takes me a bit of time to understand this complex machinery :)</p>



<a name="197180137"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Improved%20span%20handling%20%2347389/near/197180137" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dylan MacKenzie (ecstatic-morse) <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Improved.20span.20handling.20.2347389.html#197180137">(May 11 2020 at 17:42)</a>:</h4>
<p>You're doing a great job so far IMO. I really appreciate your work on <a href="https://github.com/rust-lang/rust/issues/70853">#70853</a></p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>